Intelligent device selection for mobile application testing

ABSTRACT

A computer-implemented facility is provided for intelligent mobile device selection for mobile application testing. The computer-implemented facility determines features of a new mobile application to be tested, and compares the features of the new mobile application with features of multiple known mobile applications to identify one or more known mobile applications with similar features. Based at least in part on automated analysis of user reviews of the one or the more known mobile applications operating in one or more types of mobile devices, the facility provides one or more risk scores for operation of the new mobile application in the one or more types of mobile devices. Further, based on the risk scores, a recommended set of mobile devices in which to test the new mobile application may be generated for use in testing the new mobile application.

BACKGROUND

Traditionally companies providing a new mobile device application testthe mobile device application to ensure that the application functionsin a desired manner on various types of mobile devices. For instance, anew mobile application may be tested on various mobile devices operatingon different platforms prior to commercialization.

In practice, an organization might obtain a large number of differentmobile devices for the purpose of testing a new mobile application.However, as the number of devices continues to increase, it is becomingmore and more expensive to obtain a significant percentage of the mobiledevices on the market upon which test a mobile application. Further,regular introduction of new device models and maintenance updates toexisting mobile devices continues to result in cost escalation fortesting new mobile applications on substantially all available devices.With the addition of new mobile device variations, such as wearabledevices, the Internet of Things (IoT) smart devices, the range of mobiledevices on which a mobile application may need to be tested foroperational verification continues to expand.

SUMMARY

The shortcomings of the prior art are overcome and additional advantagesare provided through the provision of a computer-implemented methodwhich includes, for instance, determining features of a new mobileapplication to be tested, and comparing, by a processor, the features ofthe new mobile application to be tested with features of multiple knownapplications to identify one or more known mobile applications withsimilar features. The method also includes, based at least in part onautomated analysis of user reviews of the one or more known mobileapplications operating in one or more types of mobile devices, providingone or more risk scores for operation of the new mobile application inthe one or more types of mobile devices. Providing the one or more riskscores includes generating a respective risk score for operating the newmobile application in each type of mobile device of the one or moretypes of mobile devices. The respective risk scores are determinedbased, at least in part, on automated analysis of user reviews of theone or more known mobile applications with similar features operating onthat type of mobile device.

Systems and computer program products relating to one or more aspectsare also described and may be claimed herein. Further, services relatingto one or more aspects are also described and may be claimed herein.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more aspects are particularly pointed out and distinctly claimedas examples in the claims at the conclusion of the specification. Theforegoing and objects, features, and advantages of one or more aspectsare apparent from the following detailed description taken inconjunction with the accompanying drawings in which:

FIG. 1 depicts an exemplary system and network which may implement oneor more aspects of the present invention;

FIG. 2 depicts one embodiment of a process implemented by an intelligentdevice selection facility, in accordance with one or more aspects of thepresent invention;

FIG. 3 depicts another embodiment of a process which may be implementedby an intelligent device selection facility, in accordance with one ormore aspects of the present invention;

FIG. 4 depicts a further embodiment of an intelligent device selectionfacility, in accordance with one or more aspects of the presentinvention;

FIG. 5 depicts a further embodiment of a computing system which mayincorporate processing, in accordance with one or more aspects of thepresent invention;

FIG. 6 depicts one embodiment of a cloud computing environment, whichmay facilitate implementing, or be used in association with, one or moreaspects of the present invention; and

FIG. 7 depicts an example of extraction model layers, which mayfacilitate implementing processing, in accordance with one or moreaspects of the present invention.

DETAILED DESCRIPTION

The description that follows includes exemplary devices, systems,methods, techniques and instruction sequences that embody techniques ofthe present invention. However, it should be understood that thedescribed invention may be practiced, in one or more aspects, withoutthe specified details. In other instances, known protocols, structuresand techniques have not been shown in detail in order not to obscure theinvention. Those of ordinary skill in the art will appreciate that thedescribed techniques and mechanisms may be applied to various systems tofacilitate intelligent selection of mobile devices upon which to test anew mobile application as described herein.

In general, a mobile device such as discussed herein, is indicative orinclusive of various types of devices, such as a wireless phone, acellular phone, a laptop computer, a wireless multimedia device, atablet, a wireless communication personal computer (PC), a personaldigital assistant (PDA), etc. Any mobile device such as reference hereinmay have memory for storing instructions and data, as well as hardware,software, and firmware combinations thereof. As is well known, mobiledevices can receive audio and/or video data packets over wirelessnetworks, and are capable of executing a wide variety of mobile deviceapplications, referred to herein as mobile applications.

As noted, the variations of mobile device offerings in the market todayis very large and, with the addition of wearable devices and Internet ofThings (IoT) smart devices, the range of devices on which to test orverify a new mobile application is expanding to unsustainable numbers.Thus, it is becoming unfeasible to plan to exhaustively test a newmobile application on every available type of mobile device in themarket place on which a user may choose to install the application.Described herein, therefore, is a novel facility for intelligentlyselecting one or more types of mobile devices upon which to test a newmobile application.

Device specific defects in mobile applications are relatively common fordifferent types of mobile devices. It can be a challenging and expensiveprocess to test a new mobile application on all available device giventhe different platforms and configurations available. One rule of thumbin the industry to solve the problem is to choose the minimum number ofmobile devices having a maximum device characteristic coverage in orderto save money. Since new mobile devices are released every day, there isno standard and trivial solution in satisfying the thumb rule. Thoughexisting approaches do help in testing new mobile applications, thereare many mobile applications released to customers today where thecustomer may subsequently complain that the application does not workproperly on a particular device.

Advantageously, disclosed herein is an intelligent device selectionfacility for mobile application testing which, in one or more aspects,may be cloud based. The intelligent device selection facility or processmay recommend a set of different types of mobile devices on which a newmobile application may be tested based, at least in part, on automatedanalysis of user reviews of one or more known mobile applications withsimilar features. In this manner, a tester may be provided with arecommended set of mobile devices upon which to test the new mobileapplication before releasing the application and the tester is providedwith a higher degree of confidence that the new mobile application willwork on most known devices in the marketplace since the new mobileapplication is to be tested on devices where similar mobile applicationswith similar features experience problems.

With reference now to the figures, and in particular to FIG. 1, a blockdiagram is depicted of an exemplary system and network that may beutilized by and/or in the implementation of one or more aspects of thepresent invention. Some or all of the exemplary architecture, includingboth depicted hardware and software, shown for and within computer 101may be utilized by application testing server 149 and may access appreviews 151.

Exemplary computer 101 includes a processor 103 that is coupled to asystem bus 105. Processor 103 may utilize one or more processors, eachof which has one or more processor cores. A video adapter 107, whichdrives/supports a display 109, is also coupled to system bus 105. Systembus 105 is coupled via a bus bridge 111 to an input/output (I/O) bus113. An I/O interface 115 is coupled to I/O bus 113. I/O interface 115affords communication with various I/O devices, including a keyboard117, a mouse 119, a media tray 121 (which may include storage devicessuch as CD-ROM drives, multi-media interfaces, etc.), and external USBport(s) 125. While the format of the ports connected to I/O interface115 may be any known to those skilled in the art of computerarchitecture, in one embodiment some or all of these ports are universalserial bus (USB) ports.

As depicted, computer 101 is able to communicate with an applicationtesting server 149 and/or other devices/systems (e.g., containing appreviews 151 of known mobile applications operating in particulardevices) using a network interface 129. Network interface 129 is ahardware network interface, such as a network interface card (NIC), etc.Network 127 may be an external network such as the Internet, or aninternal network such as an Ethernet or a virtual private network (VPN).In one or more embodiments, network 127 is a wireless network, such as aWi-Fi network, a cellular network, etc.

A hard drive interface 131 is also coupled to system bus 105. Hard driveinterface 131 interfaces with a hard drive 133. In one embodiment, harddrive 133 populates a system memory 135, which is also coupled to systembus 105. System memory is defined as a lowest level of volatile memoryin computer 101. This volatile memory includes additional higher levelsof volatile memory (not shown), including, but not limited to, cachememory, registers and buffers. Data that populates system memory 135includes operating system (OS) 137 and application programs 143.

OS 137 includes a shell 139, for providing transparent user access toresources such as application programs 143. Generally, shell 139 is aprogram that provides an interpreter and an interface between the userand the operating system. More specifically, shell 139 executes commandsthat are entered into a command line user interface or from a file.Thus, shell 139, also called a command processor, is generally thehighest level of the operating system software hierarchy and serves as acommand interpreter. The shell provides a system prompt, interpretscommands entered by keyboard, mouse, or other user input media, andsends the interpreted command(s) to the appropriate lower levels of theoperating system (e.g., a kernel 141) for processing. While shell 139may be a text-based, line-oriented user interface, the present inventionwill equally well support other user interface modes, such as graphical,voice, gestural, etc.

As depicted, OS 137 also includes kernel 141, which includes lowerlevels of functionality for OS 137, including providing essentialservices required by other parts of OS 137 and application programs 143,including memory management, process and task management, diskmanagement, and mouse and keyboard management.

Application programs 143 include a renderer, shown in exemplary manneras a browser 145. Browser 145 includes program modules and instructionsenabling a world wide web (WWW) client (i.e., computer 101) to send andreceive network messages to the Internet using hypertext transferprotocol (HTTP) messaging, thus enabling communication with applicationtesting server 149 and one or more other systems containing user reviewsof known applications (i.e., app reviews 151).

Application programs 143 in computer 101 system memory also includeslogic 147 for user-review-based (i.e., intelligent) device selection formobile application testing, in accordance with one or more aspects ofthe present invention. This logic may include code for implementing theprocesses described herein, including (for example) those describedbelow with reference to FIGS. 2-4. In one embodiment, computer 101 maybe able to download logic 147 from, for instance, application testingserver 149, including in an on-demand basis, wherein the code is notdownloaded until needed for execution. In another embodiment of thepresent invention, application testing server 149 may perform all of thefunctions associated with the present invention (including execution oflogic 147), thus freeing computer 101 from having to use its owninternal computing resources to execute logic 147.

In one or more other embodiments, computer 101, with logic 147 foruser-review-based device selection for mobile application testing, andapplication server 149 may be implemented in a common system, which mayalso include or access a database containing user reviews of knownmobile applications (app reviews 151) for use as described herein.

Further, note that the hardware elements depicted in computer 101 arenot intended to be exhaustive. For instance, computer 101 may includealternate memory storage devices such as magnetic cassettes, digitalversatile disks (DVDs), Bernoulli cartridges, and the like. These andother variations are intended to be within the spirit and scope of thepresent invention.

As noted, provided herein is an intelligent device selection process formobile application testing. End users who experience defects in knownmobile applications often publicize the defects in one or more app storereviews that they post. Given a statistically significant number of endusers, the information contained in these app store reviews can form acorpus of valuable data about which devices are experiencing defects fora given feature of a known mobile application. Disclosed herein is afacility to parse the user review text and capture the information inthe reviews that correlates a mobile application feature across multipledifferent mobile device types in order to narrow down which types ofdevices are the most important mobile devices on which to test a newmobile application using such features.

In one or more aspects, the facilities described herein leverage a crowdsourced set of data that includes many different applications, users anddevices, and pulls from that large data set that mobile devices thatappear to be the most problematic for a selected mobile applicationfeature (e.g., selected software component). If the new application tobe tested contains a feature (e.g., leverages a capability) that showsas “weak” or often exhibiting defects for one or more mobile devices,then the recommended set of mobile devices upon which to test the newmobile application may include that devices where other users havereported problems with similar features in known mobile applications.Advantageously, where a large number of users of a particular mobiledevice type across multiple mobile applications report the same (orsimilar) failure or defect with a certain feature or capability, thenany new mobile application using that same feature or capability shouldat least be tested on those devices where users have reported problems.This concept is elaborated with the addition of risk scores beingdetermined for further refinement of the available informationcontained, for instance, in the app store review text of the userreviews.

FIG. 2 depicts one embodiment of processing of an intelligent deviceselection facility for mobile device application testing, in accordancewith one or more aspects of the present invention. In one or moreembodiments, the processing illustrated is a computer-implementedprocess, executed, for instance, by one or more processors. As shown,the process may include determining features of a new mobile applicationto be tested 200, and comparing the features of the new mobileapplication to be tested with features of multiple known mobileapplications to identify one or more known mobile applications withsimilar features 210. As noted, based at least in part on automatedanalysis of user reviews of one or more known mobile applicationsoperating in one or more types of mobile devices, the process includesproviding one or more risk scores for operation of the new mobileapplication in the one or more types of mobile devices 220. Further, theprocess may include using, at least in part, the one or more risk scoresin deciding which types of mobile devices to test operation of the newmobile application in 230. For instance, a recommended set of mobiledevices to test the new application on may be provided 240, and the newmobile application may be tested at least on the recommended set ofmobile device types 250.

In one or more aspects, the intelligent device selection process mayutilize a capability to learn about user reviews from one or moregeneric app stores (such as, online stores that provide mobileapplications to consumers and publish reviews about the applicationsfrom the consumers). For instance, a representation learner (that is,heuristic logic that is able to evaluate user reviews), may receivegeneric app store reviews. The representation learner may output arepresentation for words found in the app store reviews, such thatsimilar words are grouped closer and dissimilar words are further apart.In one or more embodiments, the representation learner may be based on aneural probabilistic language model (NPLM), which understands thecontext of words in an app review. Since the input to the representationlearner is (for instance) the app store reviews, words used in thereviews may be captured, and a correlation between the words may befound. This modeling approach may become more accurate as more reviewsare input to the representation learner, since more data points meanbetter correlations and information. This process may includecategorizing reviews as either positive reviews containing praise for amobile application, or negative reviews having complaints about a mobileapplication. A computer-implemented sentiment classifier may be trainedto recognize positive verses negative reviews, such as by recognizingcertain key words (e.g., awful, bad, disappointing, etc.) for negativereviews and other key words (e.g., great, useful, valuable, etc.) forpositive reviews.

Additionally, reviews may be classified based on an associated rating(e.g., “5” for the best and “1” for the worst), with outputs from theclassifier being predictions as to whether the user reviews werepositive or negative. Further, negative reviews may be classified indifferent types of reviews, such as reviews directed to thefunctionality of the mobile application, the performance of the mobileapplication, the usability of the mobile application, etc. Theclassifier may be trained to classify negative reviews into one or moreof these review types, and may appropriately tag the reviews which canadvantageously be used to associate or identify features of the knownmobile applications with the reviews. Thus, a deep learning basedapproach may be employed to analyze application store reviews related toone or more known applications to semantically identify defects,prioritize defects and classify defects to, for instance, facilitate thecapabilities described herein.

One such system is the Watson™ system available from InternationalBusiness Machine Corporation of Armonk, N.Y. The Watson™ system is anapplication of advanced natural language processing, informationretrieval, knowledge representation and reasoning, and machine-learningtechnologies in the field of open domain question answering. The Watson™system is built on International Business Machine Corporation's DeepQA™technology used for hypothesis generation, massive evidence gathering,analysis, and scoring.

By way of further example, FIG. 3 depicts another embodiment of anintelligent device selection process for mobile application testing, inaccordance with one or more aspects of the present invention. Theprocess includes analyzing user reviews of known mobile applications andfiltering for defects of known mobile applications in specific types ofmobile devices 300. For a known mobile application in the app store, theuser reviews of the application may be semantically parsed and filteredto identify reviews which discuss failures or defects of applications inspecific types of mobile devices. The user reviews could be collectedfrom an app store, a stack overflow, Twitter™, or any other public datasource. By doing semantic analysis on the user reviews, the problematicmobile device types for a particular known mobile application may beidentified, and the feature from the reviews which led to the defectdiscovery may also be determined 310.

For a new mobile application to be tested, the known mobile applicationswith one or more similar features are identified 320. For instance, anapplication developer may provide a requirement and feature document forthe new mobile application to the intelligent device selection processor facility described herein, which in one or more implementations, maybe implemented as a cloud service. The cloud service could determinesimilar applications in the app store(s) having identical features andrequirements.

For each known application with similar features, processing determinesor identifies any problem mobile device, and a risk score for operatingthe new application on that mobile device 330. In one or moreembodiments, the risk score may be determined from one or moreparameters, such as a degree of feature similarity between the newmobile application and the known mobile application with similarfeature(s), a number of defect instances reported for the known mobileapplication on the particular type of mobile device, a sentiment of userreviews of the known mobile device application on the particular type ofmobile device, and a number of like or dislike reviews of the knownmobile application on the particular type of mobile device. In oneembodiment, the risk score may be based on multiple such parameters, forinstance, may be based on a combination of each of the noted parameters.Alternatively, other parameters could be employed to derive the score.The different mobile device types may be sorted based on the ascertainedrisk scores, and a recommended set of devices may be provided on whichto test the new application 340. The recommended set of mobile deviceswould include those mobile devices where the known mobile applicationwith similar features may have failed or been otherwise problematic. Atester may test the new application on at least the recommended set ofmobile device types 350.

Advantageously, the process described results in a higher chance ofdiscovering a problem with a new mobile application by testing thatmobile application on the mobile device types with which the knownmobile applications with similar features have had problems. Forinstance, where a known mobile application with the same feature hasexperienced a problem on a particular mobile device type, then there isa reasonable chance that another, new mobile application implementingthe same feature may also experience a problem. Thus, in one or moreaspects, disclosed herein is processing for deriving a mapping betweenapplication features and problematic devices by mining natural text inuser feedback reviews of published applications (e.g., in an app store,or other public place) for suggesting mobile device types to test thenew mobile application on based on feature matching and the derivedmapping from the user reviews of known mobile applications, forinstance, to derive a risk score which may then be used to select theset of mobile device types on which to test the new mobile application.

FIG. 4 depicts a high level system 400 implementing one or more aspectsof the present invention. In this embodiment, system 400 is implementedwithin or includes a cloud based service 410 having an applicationidentification component 412. In one or more implementations,requirement and feature documentation for the new mobile application tobe tested 401 may be provided via a software requirement document 420.The application identification component 412 reads the softwarerequirement document to identify the category and features of the mobileapplication under test 401. Application identification component 412retrieves the known mobile applications from the mobile app store(s) 430that are in the same category and also collects the features of themobile applications from the app store(s). From the list of known mobileapplications retrieved from the app store(s) 430, applicationidentification component 412 determines the known mobile applicationsthat have similar features and requirements as the new mobileapplication (i.e., the application under test 401). The applications inthe same category with identical or similar features 414 may be providedto a review analytics and risk score determination component 416 foranalytical review and determination of a risk score in order to generatea device list 440 including one or more types of mobile devices uponwhich the new mobile application is to be tested.

More particularly, for each known mobile application identified withidentical or similar features 414, the review analytics and risk scoredetermination component 416 analyzes the reviews of that known mobileapplication and filters the reviews which identify or discuss failuresor defects of the application in specific types of mobile devices. Forinstance, “My app crashed in mobile device 1 when map is rendered” or“Mobile device 2 got hung when downloading huge data”. Semantic analysisof the available user reviews may be performed on the review comments.Defect mining from application store reviews may proceed as describedabove, in one or more implementations. By performing semantic analysis,the review and risk score component determines the problematic mobiledevice types for the new mobile application, and determines the featuresfrom the reviews which lead to the discovery of the defect respectivethat mobile device type. A risk score can be calculated for the newmobile application to operate on that particular mobile device typebased on a variety of parameters, including, for instance, the degree ofrequirements/feature similarity, number of instances reported, sentimentof the reviews, and like/dislike of the user review. The discoveredknown mobile applications are sorted based on their risk score and theproblematic mobile device types for those applications are collected ina list to be provided to the tester. In this manner, the tester or newapplication developer has a higher chance of discovering more defects bytesting the new mobile application in the more problematic mobile devicetypes.

Exemplary embodiments of further computing environments to implement oneor more aspects of the present invention are described below withreference to FIGS. 5-7.

By way of further example, FIG. 5 depicts one embodiment of a computingenvironment 500, which includes a computing system 512. Examples ofwell-known computing systems, environments, and/or configurations thatmay be suitable for use with computer system 512 include, but are notlimited to, a desktop computer, a workstation, a handheld or laptopcomputer or device, a mobile phone, a programmable consumer electronicdevice, a tablet, a personal digital assistant (PDA), and the like.

Computing system 512 may be described in the general context of computersystem-executable instructions, such as program modules, being executedby a computer system. Generally, program modules may include routines,programs, objects, components, logic, data structures, and so on thatperform particular tasks or implement particular abstract data types.

As depicted in FIG. 5, computing system 512, is shown in the form of ageneral-purpose computing device. The components of computing system 512may include, but are not limited to, one or more processors orprocessing units 516, a system memory 523, and a bus 518 that couplesvarious system components including system memory 523 to processor 516.

In one embodiment, processor 516 may be based on the z/Architecture®offered by International Business Machines Corporation, or otherarchitectures offered by International Business Machines Corporation orother companies. z/Architecture® is a registered trademark ofInternational Business Machines Corporation, Armonk, N.Y., USA. Oneembodiment of the z/Architecture® is described in “z/Architecture®Principles of Operation,” IBM Publication No. SA22-7832-10, March 2015,which is hereby incorporated herein by reference in its entirety.

In other examples, it may be based on other architectures, such as thePower Architecture offered by International Business MachinesCorporation. One embodiment of the Power Architecture is described in“Power ISA™ Version 2.07B,” International Business Machines Corporation,Apr. 9, 2015, which is hereby incorporated herein by reference in itsentirety. POWER ARCHITECTURE is a registered trademark of InternationalBusiness Machines Corporation, Armonk, N.Y., USA. Other names usedherein may be registered trademarks, trademarks, or product names ofInternational Business Machines Corporation or other companies.

Bus 518 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnect (PCI) bus.

Computing system 512 may include a variety of computer system readablemedia. Such media may be any available media that is accessible bycomputing system 512, and it includes both volatile and non-volatilemedia, removable and non-removable media.

System memory 523 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 530 and/or cachememory 532. Computing system 512 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 534 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media could be provided.In such instances, each can be connected to bus 518 by one or more datamedia interfaces. As described below, memory 523 may include at leastone program product having a set (e.g., at least one) of program modulesthat are configured to carry out the functions of embodiments of theinvention.

Program/utility 540, having a set (at least one) of program modules 542,may be stored in memory 532 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 542 generally carry out the functionsand/or methodologies of embodiments of the invention as describedherein. Alternatively, a separate, device selection for app testingmodule, logic, etc., 501 may be provided within computing environment512.

Computing system 512 may also communicate with one or more externaldevices 514 such as a keyboard, a pointing device, a display 524, etc.;one or more devices that enable a user to interact with computing system512; and/or any devices (e.g., network card, modem, etc.) that enablecomputing system 512 to communicate with one or more other computingdevices. Such communication can occur via Input/Output (I/O) interfaces522. Still yet, computing system 512 can communicate with one or morenetworks such as a local area network (LAN), a general wide area network(WAN), and/or a public network (e.g., the Internet) via network adapter520. As depicted, network adapter 520 communicates with the othercomponents of computing system, 512, via bus 518. It should beunderstood that although not shown, other hardware and/or softwarecomponents could be used in conjunction with computing system 512.Examples, include, but are not limited to: microcode, device drivers,redundant processing units, external disk drive arrays, RAID systems,tape drives, and data archival storage systems, etc.

One or more aspects may relate to or use cloud computing.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of certainteachings recited herein are not limited to a cloud computingenvironment. Rather, embodiments of the present invention are capable ofbeing implemented in conjunction with any other type of computingenvironment now known or later developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based email). Theconsumer does not manage or control the underlying cloud infrastructureincluding network, servers, operating systems, storage, or evenindividual application capabilities, with the possible exception oflimited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

A cloud computing node may include a computer system/server, such as theone depicted in FIG. 5. Computer system/server 512 of FIG. 5 may bepracticed in distributed cloud computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed cloud computing environment,program modules may be located in both local and remote computer systemstorage media including memory storage devices. Computer system/server512 is capable of being implemented and/or performing any of thefunctionality set forth hereinabove.

Referring now to FIG. 6, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 6 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring to FIG. 7, a set of functional abstraction layers provided bycloud computing environment 50 is shown. It should be understood inadvance that the components, layers, and functions shown in FIG. 7 areintended to be illustrative only and embodiments of the invention arenot limited thereto. As depicted, the following layers and correspondingfunctions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and intelligent device selection app testingprocessing 96.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinaryskills in the art without departing from the scope and spirit of thedescribed embodiments. The terminology used herein was chosen to bestexplain the principles of the embodiments, the practical application ortechnical improvement over technologies found in the marketplace, or toenable others of ordinary skills in the art to understand theembodiments disclosed herein.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

In addition to the above, one or more aspects may be provided, offered,deployed, managed, serviced, etc. by a service provider who offersmanagement of customer environments. For instance, the service providercan create, maintain, support, etc. computer code and/or a computerinfrastructure that performs one or more aspects for one or morecustomers. In return, the service provider may receive payment from thecustomer under a subscription and/or fee agreement, as examples.Additionally or alternatively, the service provider may receive paymentfrom the sale of advertising content to one or more third parties.

In one aspect, an application may be deployed for performing one or moreembodiments. As one example, the deploying of an application comprisesproviding computer infrastructure operable to perform one or moreembodiments.

As a further aspect, a computing infrastructure may be deployedcomprising integrating computer readable code into a computing system,in which the code in combination with the computing system is capable ofperforming one or more embodiments.

As yet a further aspect, a process for integrating computinginfrastructure comprising integrating computer readable code into acomputer system may be provided. The computer system comprises acomputer readable medium, in which the computer medium comprises one ormore embodiments. The code in combination with the computer system iscapable of performing one or more embodiments.

Although various embodiments are described above, these are onlyexamples. For example, computing environments of other architectures canbe used to incorporate and use one or more embodiments. Further,different instructions, instruction formats, instruction fields and/orinstruction values may be used. Many variations are possible.

Further, other types of computing environments can benefit and be used.As an example, a data processing system suitable for storing and/orexecuting program code is usable that includes at least two processorscoupled directly or indirectly to memory elements through a system bus.The memory elements include, for instance, local memory employed duringactual execution of the program code, bulk storage, and cache memorywhich provide temporary storage of at least some program code in orderto reduce the number of times code must be retrieved from bulk storageduring execution.

Input/Output or I/O devices (including, but not limited to, keyboards,displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives andother memory media, etc.) can be coupled to the system either directlyor through intervening I/O controllers. Network adapters may also becoupled to the system to enable the data processing system to becomecoupled to other data processing systems or remote printers or storagedevices through intervening private or public networks. Modems, cablemodems, and Ethernet cards are just a few of the available types ofnetwork adapters.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise” (andany form of comprise, such as “comprises” and “comprising”), “have” (andany form of have, such as “has” and “having”), “include” (and any formof include, such as “includes” and “including”), and “contain” (and anyform contain, such as “contains” and “containing”) are open-endedlinking verbs. As a result, a method or device that “comprises”, “has”,“includes” or “contains” one or more steps or elements possesses thoseone or more steps or elements, but is not limited to possessing onlythose one or more steps or elements. Likewise, a step of a method or anelement of a device that “comprises”, “has”, “includes” or “contains”one or more features possesses those one or more features, but is notlimited to possessing only those one or more features. Furthermore, adevice or structure that is configured in a certain way is configured inat least that way, but may also be configured in ways that are notlisted.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to best explain theprinciples of one or more aspects of the invention and the practicalapplication, and to enable others of ordinary skill in the art tounderstand one or more aspects of the invention for various embodimentswith various modifications as are suited to the particular usecontemplated.

What is claimed is:
 1. A computer-implemented method comprising:determining features of a new mobile application to be tested;comparing, by a processor, the features of the new mobile application tobe tested with features of multiple known mobile applications toidentify one or more known mobile applications with similar features;and based at least in part on automated analysis of user reviews of theone or more known mobile applications operating in one or more types ofmobile devices, providing one or more risk scores for operation of thenew mobile application in the one or more types of mobile devices,wherein providing the one or more risk scores comprises generating arespective risk score for operating the new mobile application in eachtype of mobile device of the one or more types of mobile devices, therespective risk scores being determined based, at least in part, onautomated analysis of user reviews of the one or more known mobileapplications with similar features operating on that type of mobiledevice.
 2. The computer-implemented method of claim 1, furthercomprising using, at least in part, the one or more risk scores indeciding which types of mobile devices to test operation of the newmobile application in.
 3. The computer-implemented method of claim 1,wherein the one or more risk scores are determined based on one or morefactors selected from a group consisting of: a degree of featuresimilarity between the new mobile application and the known mobileapplication with similar features, a number of defect instances reportedfor the known mobile application on the type of mobile device, asentiment of user reviews of the known mobile device application on thetype of mobile device, and a number of like or dislike reviews of theknown mobile application on the type of mobile device.
 4. Thecomputer-implemented method of claim 1, further comprising automatedlyanalyzing user reviews of the multiple known mobile device applicationsoperating in the one or more types of mobile devices, and filtering theuser reviews to identify any technical issue with a known mobile deviceapplication on a particular type of mobile device, the technical issuerelating to using of the known mobile device application on thatparticular type of mobile device.
 5. The computer-implemented method ofclaim 4, wherein the automatedly analyzing is based on semantic analysisof the user reviews of the multiple known mobile applications.
 6. Thecomputer-implemented method of claim 1, wherein the features of the newmobile application include features selected from the group consistingof operational requirements for the new mobile application andattributes of the new mobile application for a user.
 7. Thecomputer-implemented method of claim 1, further comprising generating arecommended set of mobile devices to test the new mobile application on,the generating being based, at least in part, on the one or more riskscores.
 8. The computer-implemented method of claim 7, furthercomprising testing the new mobile application on the recommended set ofmobile devices.